{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![LOGO](../../../img/MODIN_ver2_hrz.png)\n",
    "\n",
    "<center><h2>Scale your pandas workflows by changing one line of code</h2>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 1: How to use Modin\n",
    "\n",
    "**GOAL**: Learn how to import Modin to accelerate and scale pandas workflows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modin is a drop-in replacement for pandas that distributes the computation \n",
    "across all of the cores in your machine or in a cluster.\n",
    "In practical terms, this means that you can continue using the same pandas scripts\n",
    "as before and expect the behavior and results to be the same. The only thing that needs\n",
    "to change is the import statement. Normally, you would change:\n",
    "\n",
    "```python\n",
    "import pandas as pd\n",
    "```\n",
    "\n",
    "to:\n",
    "\n",
    "```python\n",
    "import modin.pandas as pd\n",
    "```\n",
    "\n",
    "Changing this line of code will allow you to use all of the cores in your machine to do computation on your data. One of the major performance bottlenecks of pandas is that it only uses a single core for any given computation. Modin exposes an API that is identical to pandas, allowing you to continue interacting with your data as you would with pandas. There are no additional commands required to use Modin locally. Partitioning, scheduling, data transfer, and other related concerns are all handled by Modin under the hood."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"text-align:left;\">\n",
    "        <h1>pandas on a multicore laptop\n",
    "    <span style=\"float:right;\">\n",
    "        Modin on a multicore laptop\n",
    "    </span>\n",
    "\n",
    "<div>\n",
    "<img align=\"left\" src=\"../../../img/pandas_multicore.png\"><img src=\"../../../img/modin_multicore.png\">\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concept for exercise: Dataframe constructor\n",
    "\n",
    "Often when playing around in pandas, it is useful to create a DataFrame with the constructor. That is where we will start.\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "frame_data = np.random.randint(0, 100, size=(2**10, 2**5))\n",
    "df = pd.DataFrame(frame_data)\n",
    "```\n",
    "\n",
    "When creating a dataframe from a non-distributed object, it will take extra time to partition the data. When this is happening, you will see this message:\n",
    "\n",
    "```\n",
    "UserWarning: Distributing <class 'numpy.ndarray'> object. This may take some time.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: Do not change this code!\n",
    "import numpy as np\n",
    "import pandas\n",
    "import sys\n",
    "import modin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pandas.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "modin.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Implement your answer here. You are also free to play with the size\n",
    "# and shape of the DataFrame, but beware of exceeding your memory!\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "frame_data = np.random.randint(0, 100, size=(2**10, 2**5))\n",
    "df = pd.DataFrame(frame_data)\n",
    "\n",
    "# ***** Do not change the code below! It verifies that \n",
    "# ***** the exercise has been done correctly. *****\n",
    "\n",
    "try:\n",
    "    assert df is not None\n",
    "    assert frame_data is not None\n",
    "    assert isinstance(frame_data, np.ndarray)\n",
    "except:\n",
    "    raise AssertionError(\"Don't change too much of the original code!\")\n",
    "assert \"modin.pandas\" in sys.modules, \"Not quite correct. Remember the single line of code change (See above)\"\n",
    "\n",
    "import modin.pandas\n",
    "assert pd == modin.pandas, \"Remember the single line of code change (See above)\"\n",
    "assert hasattr(df, \"_query_compiler\"), \"Make sure that `df` is a modin.pandas DataFrame.\"\n",
    "\n",
    "print(\"Success! You only need to change one line of code!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have created a toy example for playing around with the DataFrame, let's print it out in different ways."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concept for Exercise: Data Interaction and Printing\n",
    "\n",
    "When interacting with data, it is very imporant to look at different parts of the data (e.g. `df.head()`). Here we will show that you can print the modin.pandas DataFrame in the same ways you would pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the first 10 lines.\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the DataFrame.\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Free cell for custom interaction (Play around here!)\n",
    "df.add_prefix(\"col\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Please move on to [Exercise 2](./exercise_2.ipynb) when you are ready**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
